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SURFACE MANAGEMENT SYSTEM DEPARTURE EVENT 

DATA ANALYSIS 


Gilena A. Monroe 
Ames Research Center 


ABSTRACT 

This paper presents a data analysis of the Surface Management System (SMS) performance of 
departure events, including push-back and runway departure events. The paper focuses on the detec- 
tion performance, or the ability to detect departure events, as well as the prediction performance of 
SMS. The results detail a modest overall detection perfonnance of push-back events and a signifi- 
cantly high overall detection performance of runway departure events. The overall detection perfor- 
mance of SMS for push-back events is approximately 55%. The overall detection performance of 
SMS for runway departure events nears 100%. This paper also presents the overall SMS prediction 
performance for runway departure events as well as the timeliness of the Aircraft Situation Display 
for Industry data source for SMS predictions. 


INTRODUCTION 

Enhanced management of air traffic and airports will achieve future goals of greater safety and effi- 
ciency in the National Airspace System (NAS) as capacity and operations increase. As the NAS 
grows, decision support tools will continue to play an integral role in the future of air traffic man- 
agement. These tools and their data sources should sustain a high level of information integrity, as 
they are used to make decisions regarding air traffic operations. 

NASA Ames Research Center and the Federal Aviation Administration (FAA) jointly developed a 
decision support tool known as the Surface Management System (SMS). SMS improves capacity, 
efficiency, and flexibility of aircraft movement on the surface of busy airports. In order to do so, 
SMS enables shared surface awareness that can be utilized by the air traffic control tower (ATCT), 
Terminal Radar Approach Control (TRACON), Air Route Traffic Control Center (Center), and air- 
line personnel through the use of map displays, timelines, and load graphs. SMS provides forecast- 
ing and advisory tools to assist in the management of surface activities (ref. 1). 

Given multiple data sources, as shown in figure 1 of the system architecture, SMS detects and pre- 
dicts both arrival and departure event times. These detections and predictions offer valuable traffic 
information on arrival or departure aircraft status, aircraft arrival or departure time for various refer- 
ence points (e.g., spot, runway), and future arrival or departure demand. This infonnation also assists 
air traffic controllers in perfonning tactical tasks such as runway assignments, as well as traffic 
management coordinators in performing strategic tasks such as runway balancing and configuration 
changes. Airline ramp controllers might also use this infonnation in gate assignment tasks. Having 
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more accurate aircraft state information and reliable predictions allows for more informed decision- 
making regarding surface operations. 

As the air traffic system evolves, so will operational procedures and the tools used to perfonn them. 
To meet that evolution process, SMS will need to provide its users with reliable data and timely, 
reliable data sources. This must be true of the system as a stand-alone tool, as well as through inte- 
gration with other air traffic management tools. 

A previous analysis of SMS arrival events showed a significant improvement in prediction accuracy 
through integration with the Traffic Management Advisor (TMA) arrival scheduler (ref. 2). The 
results indicated that the integrated system produced more accurate runway arrival times than the 
stand-alone SMS system throughout the prediction horizon (ref. 3). No such analysis of SMS has 
been done specifically for departure events. 

The goal of this analysis was to determine the overall detection and accuracy performance of the 
SMS system as well as the timeliness and accuracy of its data sources, specifically for departure 
events. The results of the departure event data analysis presented here will help to identify areas of 
improvement in departure event detection and prediction. This analysis will also help illustrate SMS 
timing behavior with its actual event data that will be integral for its planned future integrations with 
other air traffic management tools. 
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Figure 1. SMS system architecture. 
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This paper focuses specifically on the detection and prediction accuracy of SMS, given its data 
sources for departure events, the Aircraft Situation Display for Industry (ASDI) and the Airport Sur- 
face Detection Equipment Model X (ASDE-X). ASDI allows near real-time air traffic data to be dis- 
tributed to the aviation industry while ASDE-X provides position and identification information of 
aircraft on the airport surface, as well as within 5 miles of the airport. This analysis used recorded 
traffic data from the Dallas/Ft. Worth International Airport (DFW). A general description of the 
push-back, or OUT, event detection capabilities and OUT event scheduling assistance capabilities 
will be provided. A description of the data sources for runway departure, or OFF event detection and 
OFF event time prediction will also be included. A comparison of the ASDI data source’s OFF time 
prediction performance and the actual SMS OFF time was completed and the results will be pre- 
sented. 


SMS BACKGROUND 

SMS is a decision support tool that manages aircraft on the surface of busy airports and within the 
terminal airspace. SMS enables shared situation awareness between both airline and FAA facilities. 
SMS also provides surface predictions and advisories to assist in the management of surface opera- 
tions and to enable air carriers and air traffic control (ATC) coordination (ref. 4). 

SMS consists of three primary toolsets to assist users. The first of these are the Controller Tools that 
use map displays, timelines, and load graphs to provide air traffic control tower and airline ramp 
controllers with flight-specific information, as well as tactical advisories for efficient traffic opera- 
tions (see figure 2). The second set of user tools are the Traffic Manager Tools that provide ATC 
Tower, TRACON, and Center managers with predictions of future demand in order to strategically 
plan future airport surface operations. The final set of tools are the NAS Information Tools that pro- 
vide surface demand predictions to NAS users for improved decision making. 



Figure 2. SMS map display, timeline, and load graph. 
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Research activity and testing of Build I SMS, a grouping of SMS capabilities, has been conducted in 
both simulation and operational environments. In 2001 and 2002, simulations were conducted to 
assess the system information requirements (ref. 5). In 2003, an operational evaluation to assess traf- 
fic management coordination usage of SMS was conducted at Memphis International Airport. Sha- 
dow testing of the traffic management functions of SMS at the Memphis Airport in 2004 assessed 
local and ground controller use of SMS. The results indicated controller acceptance of the display 
tools, as well as the accuracy of the displayed data on SMS (ref. 6). The results also showed the need 
for improved prediction accuracy (ref. 1). 

Currently, SMS is fielded in various locations throughout the nation. It is operational at the both the 
Federal Express and Northwest Airlines Ramp Towers at the Memphis International Airport as well 
as at the United Parcel Service facilities at Louisville International Airport. It is also installed at the 
center tower of the DFW airport to support NASA research. SMS is will also be used in future 
NASA research as an engineering testbed for surface simulations. 


SMS PREDICTION CAPABILITY 

Using multiple data sources, SMS possesses predictive capabilities to assist in the scheduling and 
routing of aircraft and in the allocation of airport resources. The predictions assist in the generation 
of advisories, which are used for managing the airport surface flow (ref. 4). SMS is able to assist air- 
line operators in scheduling push-back events to improve flight prioritization, as well as predict OFF 
events. 


A. Push-Back (OUT) Scheduling 

The OUT time delineates when an aircraft pushes back from its gate. Currently, flights are autho- 
rized to push back from the gate when they notify the ramp controller that they are ready for push- 
back. SMS predicts the amount of taxi delay that each aircraft may incur once it has pushed back 
and can advise ramp controllers to modify their push-back procedures to minimize the total taxi 
delay of the aircraft. 

Accurate push-back time prediction is beneficial to airport surface operations. Estimates of actual 
push-back times could allow for faster and more accurate runway departure time predictions (ref. 7). 
When SMS predicts a taxi delay, and a possibility of waiting in a departure queue when an aircraft 
indicates readiness for an early push-back, SMS may advise ramp controllers to temporarily hold the 
push-back of the aircraft. By holding the push-back time of these flights, other late operating push- 
back aircraft may be given priority and placed ahead of these early flights, thus improving overall 
efficiency. Aircraft wait time in taxi queues may also be minimized, which may reduce the cost to 
the airlines through reduced fuel use. 

Flight plan data containing scheduled push-back times are provided to SMS by ASDI (ref. 1). Unlike 
runway departure predictions, which receive updated aircraft track data throughout the taxi status of 
the aircraft, the push-back prediction is largely uncertain since the only input to this prediction is the 
flight plan push-back time. With the inclusion of airline data, which provides aircraft gate informa- 
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tion, estimated push-back time, and flight status, SMS can better predict push-back times, gate arriv- 
al times, and takeoff times. Also airport terminal area activities (i.e., passenger delay, catering, and 
maintenance) introduce uncertainty in push-back times, which would greatly affect any predictions. 
Although SMS does not provide precise OUT time predictions in the absence of airline data, it is 
capable of assisting ramp operators with scheduling OUT events with the knowledge of predicted 
taxi delay. 


B. Runway Departure (OFF) Event Prediction 

The OFF time delineates the time at which an aircraft starts its take-off roll from its designated run- 
way. The prediction time of an OFF event is calculated based on when an aircraft pushes back from 
its gate, the amount of taxi time and delay incurred, and the time spent in the departure queue. 
Depending upon the status of an aircraft, along with the availability of other data sources, SMS can 
predict OFF times for flights using three methods: Enhanced Traffic Management System (ETMS)/ 
ASDI data only (i.e., planned push-back times from the Collaborative Decision Making data stream, 
which is part of ETMS), planned push-back data as well as real-time push-back event data, or 
planned push-back data with complete real-time surface surveillance data (ref. 1). 

The large uncertainty in aircraft OFF time predictions impacts all other predictions downstream. By 
modeling the movement of the traffic that is actually on the surface of the airport and, thereby, sup- 
plying accurate taxi time estimates, SMS can further improve the time predictions of OFF events. 
These SMS-predicted OFF times can then be used to improve all other predictions and products that 
are based on predicted OFF times such as regional traffic flow management, including arrival meter- 
ing scheduling. 


DATA ANALYSIS 

The data used in these analyses were taken from the Dallas-Fort Worth International Airport for 
departure events only. Data were collected over a period of five days, from January 22-26, 2007, 
during the three major departure pushes: 8:00 a.m., 1 1:00 a.m., and 1:30 p.m. Central Standard 
Time, when there is a higher capacity of departure operations than at other times of the day. The data 
collected from those days were recorded for one-hour time durations and were averaged over the 
five days for the reported results. The data were collected under normal traffic and weather condi- 
tions with a North departure flow configuration. No anomalies were removed from the data sets, and 
each reported value reflects the actual data collected. ASDI and ASDE-X data were used as sources 
for event detection and prediction time. Though airline data may also be used as a source for event 
detection and prediction time, it was not available when these analyses were conducted. Here, an 
accurate prediction of detection and prediction time is defined as a time error within a -1 to 1 minute 
range. 
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A. OUT Event 


Detection 

The accurate detection of push-back events can help to improve departure event predictions. SMS 
uses surface surveillance data to detect push-back events by examining both the position and speed 
of an aircraft while it is in the ramp area. SMS uses model output AOUT (actual OUT) data, a col- 
lection of data variables, to identify an actual OUT event of an aircraft. SMS also considers an air- 
craft to have pushed back when the aircraft is first detected on the movement area of the airport; 
SMS then “guesses” and assigns its gate push-back time to be the same as the time it was actually 
detected. SMS denotes these occurrences as guessed detections. This OUT event detection analysis 
determines the number of push-back events that actually occur during 15-minute intervals over 
1-hour durations. Because cooperative surveillance data, such as multi-lateration, is used in the 
detection of push-back events, this analysis gives insight into the number of aircraft that have their 
transponders on before leaving the ramp area. 


B. OFF Event 


Detection 

Just as surface surveillance data are used to detect push-back events, they are also used to detect 
runway departure events of aircraft on the surface of the airport. SMS uses model output AOFF 
(actual OFF) data, a collection of data variables, to identify an actual off event occurrence of an air- 
craft. SMS also considers an aircraft to have departed the runway when it is first detected, either on 
the surface or within the tenninal airspace, and calculates or “guesses” the runway departure time of 
the aircraft. SMS denotes these events as guessed detections. This OFF event detection analysis 
determines the number of runway departure events that actually occur during 15 -minute intervals 
over 1-hour durations. 

Prediction accuracy 

As SMS predicts the OFF time of an aircraft, it uses a timestamp from the GUIFLIGHT data, a col- 
lection of controlled and undelayed timestamps for reference points on the surface and in the termin- 
al airspace, to provide predictions to the system. The timestamps from these data are matched with 
the model output AOFF data to identify the total number of matching timestamps between the 
GUIFLIGHT and AOFF data sets. This OFF event prediction accuracy analysis detennines the over- 
all prediction accuracy, within 1 minute of the actual event, of runway departure events. 

ASDI data source comparison 

SMS receives data input from various sources, which in turn affect the timeliness as well as accura- 
cy of the system. SMS receives the DZ message, a transmitted departure message for all eligible 
activated flight plans, from its ASDI data source to supply the system with runway departure times 
for aircraft (ref. 8). The timestamps from the DZ messages are compared with the SMS model AOFF 
timestamps to determine the difference in minutes for the arrival of the OFF event data. This OFF 
event ASDI data source comparison analysis compares the timeliness of the ASDI data with the 
actual time of the OFF event. 
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RESULTS 


A. OUT Event 


Overall detection 

The data analysis was performed on the overall OUT event detection performance of SMS. When 
SMS detects that an aircraft has pushed back from the gate, via surface surveillance data, an actual 
push-back detection is identified. If an aircraft is detected on the surface after it has actually pushed 
back from its gate, SMS will “guess” its OUT time. For example, if an aircraft is scheduled to push- 
back from its gate at 9: 15 a.m., but it is not detected on the surface until 9:20 a.m. and it is already 
taxiing, SMS will identify the aircraft as having pushed back and will “guess” the OUT time of the 
aircraft to be 9:20 a.m., even though its current location is on a taxiway. SMS assigns the current 
time for an OUT event, because there is no way to accurately back calculate an OUT time of an air- 
craft after it has left its gate. This is due to the uncertainty of the actions of the aircraft before it is 
actually detected. For instance, an aircraft may have been sitting in the ramp area for an extended 
amount of time or may have had to return to its gate after departing. Because the system is unaware 
of any of these potential events, a truthful OUT time calculation cannot be provided. 

Figure 3 shows the number of flights that were detected or guessed by the SMS system over three 
hours of departure pushes at the DFW airport. The data are shown in 15 -minute intervals from the 
start of the departure push hour. Figure 3 also shows the percentage of actual detections made by 
SMS, not including the guessed detections. 

As the figure shows, a little more than half of the departure flights were identified as actual surface 
surveillance detections. There were no significant differences at the various rush periods. These data 
reveal room for improvement in the detection of OUT events within the SMS system, especially dur- 
ing heavy departure traffic times. These improvements, though, are greatly dependent upon the air- 
craft capabilities and the undelayed activation of transponders by pilots. 

SMS uses aircraft position and speed to detect push-back events, but there are times when SMS does 
not have this infonnation. One major reason is pilot delay in turning on the transponder, which 
projects the aircraft position data for the surface surveillance to identify it. If the transponder is not 
turned on before the aircraft pushes back from its gate, it is possible that an aircraft does not get 
detected until it is taxiing or even until it is departing its assigned runway. By increasing transponder 
usage at the gate, OUT event detection within SMS will improve. As a result, OFF time predictions 
would be improved and lead to more efficient surface and en-route operations. 
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Figure 3. OUT event detection performance. 


B. OFF Event 


Overall detection 

This data analysis was performed on the overall OFF event detection performance of SMS. When 
SMS detects that an aircraft has departed its assigned runway, via surface surveillance data, an 
actual runway departure is identified. If an aircraft is detected after it has actually started its roll for a 
runway departure, SMS will calculate, or “guess,” the OFF time of the aircraft, dependent upon its 
current location. SMS will identify this as a guessed departure. For example, if an aircraft is sche- 
duled to depart its runway at 2:00 p.m., but it is not detected until it has already left the runway, 

SMS will identify the aircraft as having departed the runway and will use the speed of the aircraft to 
back calculate its runway departure time and “guess” the OFF time of the aircraft to be 2:03 p.m., 
since its current location is beyond the runway. 

Figure 4 shows the number of flights that were detected or guessed by the SMS system over three 
hours of departure pushes at the DFW airport. The data are shown in 15-minute intervals from the 
start of the departure push hour. Figure 4 also shows the percentage of actual detections made by 
SMS, not including the guessed detections. 
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Figure 4. OFF event detection rerformance. 


As shown in figure 4, the results of the analysis indicate that SMS is successful in detecting runway 
departure events. For each of the 15-minute intervals over the 3 hours of operations, figure 4 shows 
100% detection, or nearly 100% detection, of the OFF events. 

Because figure 3 indicates that many transponders are not activated while still at the gate, it appears 
that they are being turned on before they reach their departing runway. The few guessed OFF events 
in figure 4 may be the result of aircraft that did not activate their transponders prior to their take-off 
roll. There were no significant differences between the various rush periods. 

Overall prediction accuracy 

The accuracy analysis was perfonned to determine the overall precision of the SMS event time pre- 
dictions for runway departures. The prediction accuracy is determined by an error rate of no more 
than 1 minute from the actual runway departure time. Since departure events typically happen within 
a 1-1 A minute timeframe, a 1- minute error rate provides high integrity to the precision of the pre- 
diction. Within the system, SMS is provided with messages from its data sources containing runway 
departure times. SMS uses these supplied departure times in its advisories and predictions that are 
displayed to the user. 


9 







Figure 5 shows the percentage of correctly predicted runway departure times by SMS, given the 
ASDI data prior to push-back and ASDE-X data after push-back. The data represent OFF time pre- 
diction accuracy for three separate departure pushes containing varying numbers of included aircraft. 
The time horizon spans from an hour prior to the actual OFF event taking place. 

Figure 5 illustrates that the farther out from the actual OFF event, SMS does not perform well in 
predicted OFF times. Until approximately 10 minutes before an OFF event, the system is constant at 
just below 10% accuracy. It is not until 10 minutes out, that prediction accuracy begins to increase, 
yet it does not reach 100% accuracy until aircraft are within approximately 1 minute of departure. 

The 10-minute timeframe before takeoff often is representative of the time taken for an aircraft to 
taxi from its gate to its departure runway at DFW. Because this is when track data are being col- 
lected for a moving aircraft, the OFF time predictions are continually updated, becoming more accu- 
rate the closer it is to the time of departure. Though inaccuracy for OFF time predictions is high 
prior to this 10-minute timeframe, this might be due to the lack of surveillance data before an aircraft 
has departed its runway. 


OffTime Prediction Correctance 



Figure 5. OFF event prediction accuracy performance. 
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Data source comparison 

The data analysis was performed on the ASDI OFF time prediction function to detennine the timeli- 
ness of the ASDI data source. ASDI provides a DZ message to SMS, which supplies it with runway 
departure times. Upon receiving this message, SMS continually updates in order to output more 
recent runway departure times. To discern how quickly the ASDI data source is supplying this in- 
formation to SMS, the ASDI information arrival time is compared to the actual OFF time given by 
the system. 

Figure 6 shows the difference in minutes between the ASDI DZ message and the actual departure 
time. These data are shown for the total number of OFF events that occurred during the 3 departure 
pushes at the DFW airport. 
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The data suggest that ASDI is providing timely runway departure times to SMS. Most of the ASDI 
DZ messages arrive to SMS within 1 minute of the system OFF time. There are outliers that mostly 
fall within the 5-6 minute range, which might be caused by data messages being missed or dropped 
out of the system during the transition from the ASDI data source to SMS. Further investigation 
would be required to evaluate this hypothesis. 


CONCLUSIONS 

It is imperative that SMS provide reliable data and timely, reliable data sources. Its detection and 
prediction accuracy, as both a stand-alone tool and an integrated tool, should be high-quality for all 
surface operations, both arrival and departure events. The goal of this analysis was to determine the 
overall detection and accuracy performance of the SMS system as well as the timeliness and accura- 
cy of its data sources, specifically for departure events. 

The results of the analysis have shown that lack of aircraft surveillance data contributes to the low 
detection performance of SMS for OUT events, while surveillance data allows nearly perfect SMS 
detection perfonnance for OFF events. 

Improvements in the accuracy of SMS push-back time predictions are anticipated to assist push-back 
scheduling that may aid the airline operators in reducing operating costs and improving flight priori- 
tization. 

Although SMS OFF time prediction accuracy is better the closer an aircraft is to its departure, SMS 
can improve OFF time prediction accuracy farther out in time through improved OUT time predic- 
tion. These improvements will likely result in NAS-wide improvement in air traffic management 
scheduling predictions. 

The analysis also shows that the ASDI data source is usually getting runway departure data to SMS 
in a timely manner. 
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RECOMMENDATIONS 


As SMS can offer valuable decision support to its users through its detection and prediction capabili- 
ties, there are a few areas of interest that might assist in the improvement of SMS functionality. 
These recommended areas are identified below. 


A. Inclusion of Airline Data 

Airline data play an integral part in the functionality of SMS. Airline data contain parking gate as- 
signments, updated predicted push-back time, flight priority, gate availability, and even maximum 
de-icing holdover time. SMS can use these data to more efficiently construct taxi routes on the sur- 
face and provide more accurate predictions of push-back and departure times as well as gate IN and 
runway arrival times. 


B. Mandatory Early Transponder Activation 

When the transponder of an aircraft is activated, it can be identified on both radar and collision 
avoidance systems. Without transponder activation, air traffic controllers may not see the aircraft on 
a radar screen. If all aircraft transponders are activated before they leave the gate area, SMS will not 
only show all aircraft on the airport surface on its map display, but will also improve on its detection 
of aircraft departure events, thus improving prediction accuracy of these events. Many pilots do not 
activate their transponders before leaving the gate area; it is not a required, standard procedure, pos- 
sibly because the aircraft at this state are usually under airline ramp rather than ATC supervision. 
Pilots may also prefer not to activate their transponders due to desires to preserve battery life. Man- 
datory, early transponder activation will prove to be beneficial to SMS prediction accuracy and sub- 
sequent surface scheduling. 


C. Surface Optimization 

The optimization of surface operations can be helpful for SMS OFF time prediction accuracy. 
Through optimized operations such as departure aircraft scheduling and taxi route planning, more 
accurate data regarding aircraft maneuvers and processes will be available to SMS. This accessibility 
to more accurate data will allow SMS to provide improved advisories and predictions to its users. 
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